Markov Chains: Ergodicity in Time-Discrete Cases

Discrete Time Markov Chains : Ergodicity Theory

Lecture 8: Discrete Time Markov Chains: Ergodicity Theory Announcements: 1. We handed out HW2 solutions and your homeworks in Friday’s recitation. I am handing out a few extras today. Please make sure you get these! 2. Remember that I now have office hours both: Wednesday at 3 p.m. and Thursday at 4 p.m. Please show up and ask questions about the lecture notes, not just the homework! No one cam...

Subgeometric ergodicity of Markov chains

When f ≡ 1, the f -norm is the total variation norm, which is denoted ‖μ‖TV. Assume that P is aperiodic positive Harris recurrent with stationary distribution π. Then the iterated kernels P(x, ·) converge to π. The rate of convergence of P(x, .) to π does not depend on the starting state x, but exact bounds may depend on x. Hence, it is of interest to obtain non uniform or quantitative bounds o...

Lecture 11: Discrete Time Markov Chains

where {Zn : n ∈ N} is an iid sequence, independent of initial state X0. If Xn ∈ E for all n ∈ N0, then E is called the state space of process X . We consider a countable state space, and if Xn = i ∈ E, then we say that the process X is in state i at time n. For a countable set E, a stochastic process {Xn ∈ E,n ∈ N0} is called a discrete time Markov chain (DTMC) if for all positive integers n ∈ ...

Discrete Time Markov Chains 1 Examples

Example 1.1 (Gambler Ruin Problem). A gambler has $100. He bets $1 each game, and wins with probability 1/2. He stops playing he gets broke or wins $1000. Natural questions include: what’s the probability that he gets broke? On average how many games are played? This problem is a special case of the so-called Gambler Ruin problem, which can be modelled using a Markov chain as follows. We will b...

Lecture 10: Discrete Time Markov Chains